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(57) A patient supported on a patient support (1 2) is 
moved into a bore (22) of a planning imaging device, 
such as a CT scanner (20). A three-dimensional diag- 
nostic image in three-dimensional diagnostic image 
space is generated and stored in a memory. Tine patient 
is repositioned outside of the bore with a region of inter- 
est in alignment with a real time imaging device, such 
as a fluoroscopic imaging device (40). A surgical plan- 
ning instrument (62), such as a pointer or biopsy needle, 
is mounted on an articulated arm (60). As the instrument 
is inserted into the region of interest, fluoroscopic imag- 
es are generated and stored in a memory (140). The 
coordinate systems of the CT scanner, the fluoroscopic 
device, and the surgical instrument are correlated such 
that the instrument is displayed on both the CT images 
(134) and the fluoroscopic images (50), such that cur- 
sors move concurrently along the fluoroscopic and CT 
innages, and the like. In a preferred embodiment in which 
the cursor on the CT image display is at an intersection 
of transverse, sagittal, and coronal planes, the dis- 
played planes change as the cursor moves coordinating 
the displayed CT image planes with the position of the 
cursor on the fluoroscopic image. 
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Description 

[0001] The present invention relates a method of, and 
apparatus for, imaging, especially for use in the field of 
medical diagnostic imaging and minimally invasive sur- 
gery. It finds particular application In conjunction with an 
integrated CT scanner, fluoroscopic x-ray device; and 
mechanical arm type minimally invasive type surgical 
tool and will be described with particular reference 
thereto. It is to be appreciated, however, that the inven- 
tion is also applicable to magnetic resonance imaging 
and other imaging systems capable of generating volu- 
metric images. The system is also applicable to other 
real time imaging systems capable of monitoring a re- 
gion of the patient during a minimally invasive surgical 
procedure. The invention is also applicable to perform- 
ing surgical procedures with minimally invasive surgical 
instruments whose position is monitored using electron- 
ic triangulation techniques rather than a mechanical 
arm. 

[0002] Heretofore, volumetric image data of patients 
have been generated using CT scanners, magnetic res- 
onance imagers, and the like. The 3D image data has 
been used to plan minimally invasive and other surgical 
procedures. However, the bulky size of the CT scanners 
and magnetic resonance imagers, the data acquisition 
time, radiation in CT scanners, strong magnetic fields in 
magnetic resonance imagers, and the like, render such 
imagers inconvenient for monitoring minimally invasive 
or other surgery in real time. More typically, a minimally 
invasive procedure might be planned based on such di- 
agnostic images and carried out at a later time. 
[0003] At one or more points during the invasive pro- 
cedure, the procedure might be stopped and the patient 
reintroduced into the imager to determine whether the 
surgical instrument has reached the target^ is following 
the appropriate trajectory, or the like. Such starting and 
stopping of the surgical procedure, movement of the pa- 
tient, and the like is again inconvenient and can intro- 
duce error. Moreover, such a system checks the surgical 
procedure after the fact and does not provide real time 
monitoring. 

[0004] Minimally invasive surgical procedures have 

been monitored using fluoroscopic projection equip- 
ment. During the interventional surgical procedure, the 
surgeon actuates the fluoroscopic equipment from time 
to time to generate projection images of the region of 
interest and the surgical instrument. However, in fluor- 
oscopic images, data from the entire subject is projected 
or compressed into a single plane. This permits the po- 
sition of the surgical device to be monitored relative to 
the two axes in the plane of the image but provides lim- 
ited cues as to the depth in the direction perpendicular 
to the image. Moreover, fluoroscopic images are typi- 
cally at relatively low radiation doses and have less res- 
olution than CT or MRI images. 
[0005] Some interventional ists display images de- 
rived from the volumetric data on one monitor concur- 



rently with the display of fluoroscopic images on another 
monitor. The data on the two images is integrated in the 
interventionalist's mind. Such mental integration raises 
the prospect of mental mistake and judgement errors. 
s [0006] In accordance with one aspect of the present 
invention, a method of diagnostic imaging Is provided. 
A region of interest of a patient is imaged using a three- 
dimensional diagnostic imaging technique to generate 
volumetric image data. As a surgical probe is inserted 
10 into the region of interest, a series of fluoroscopic pro- 
jection images of the region of interest and the instru- 
ment are generated to monitor movement of the surgical 
instrument. The volumetric image data is generated and 
displayed in a volumetric image coordinate system; and 
15 the fluoroscopic images are generated and displayed in 
a fluoroscopic image coordinate system. The fluoro- 
scopic image coordinate system and the volumetric im- 
age coordinate system are electronically correlated. 
[0007] In accordance with another aspect of the 
20 present invention, a medical diagnostic imaging system 
is provided. A surgical planning instrument is movable 
by an operator to any of a plurality of locations adjacent 
to and in a region of interest of a subject. A system gen- 
erates electrical signals indicative of coordinates of the 
25 instrument in instrument space. A three-dimensional di- 
agnostic imaging apparatus generates three-dimen- 
sional diagnostic image data which is indicative of ana- 
tomical structure of the subject in three-dimensional im- 
age space. A three-dimensional diagnostic image mem- 
30 ory stores the generated three-dimensional diagnostic 
image data. A real time imaging device generates real 
time image data of the region of interest in real time im- 
age space. A transform processor transforms at least 
one coordinate of the surgical instrument into the three 
35 dimensional image space. At least one human-readable 
display is connected with the transform processor, the 
three-dimensional image data memory, and the real 
time imaging device for generating a human -readable 
display of a real time image and at least one of (i) slices 
40 of the three-dimensional imaging data which slices in- 
tersect at the transformed coordinate and (ii) an image 
generated from the three-dimensional image data with 
a representation of the surgical instrument superim- 
posed thereon. 

45 [0008] Ways of carrying out the invention will now be 
described in detail, by way of example, with reference 
to the accompanying drawings, in which: 

Figure 1 is a diagrammatic illustration of a CT scan- 
50 ner, fluoroscopic system, and minimally invasive 
surgical arm in accordance with the present inven- 
tion; 

Figure 2 is a diagrammatic illustration of real time 
55 and planning image processing performed with the 
apparatus of Figure 1 ; and 

Figure 3 is a diagrammatic illustration illustrative of 
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resolving three-dimensional position information 
from a pair of angularly offset fluoroscopic images. 

[0009] With reference to Figure 1, a patient table or 
support 10 includes a patient supporting surface 12 that 

is mounted for longitudinal movement relative to a base 
portion 14. The base portion 14 includes a motor for rais- 
ing and lowering the patient support surface 12 and for 
moving the patient support surface longitudinally. Posi- 
tion encoders are also provided for generating electrical 
signals indicative of the height and longitudinal position 
of the patient support. The patient support includes a 
calibration marker 16 disposed at a known, fixed loca- 
tion. 

[0010] A planning, preferably volumetric diagnostic 

imaging apparatus 20 is disposed in axial alignment with 
the patient table such that a patient or subject on the 
patient support surface 12 can be moved into and 
through a bore 22 of the volumetric imager. In the illus- 
trated embodiment, the volumetric imager is a CT scan- 
ner which includes an x-ray tube mounted for rotation 
about a preselected plane. The x-ray tube projects a fan- 
shaped beam of radiation through a ring 24 of radiation 
translucent material, through the patient support 12, 
through a region of interest of the subject, and to a ring 
or arc of radiation detectors positioned opposite the x- 
ray tube. As the x-ray tube rotates within the plane, a 
series of data lines are generated, which data lines are 
reconstructed into at least a slice image by a reconstruc- 
tion processor included in a control console 30. More 
specifically to the preferred embodiment, the patient 
support 12 moves longitudinally as the x-ray tube is ro- 
tating around the subject such that a selected volume 
of the patient is scanned along a spiral path or a series 
of slices. The position of the x-ray tube is monitored by 
a rotational position encoder and the longitudinal posi- 
tion of the patient support as monitored by a longitudinal 
position encoder within the couch 10. The reconstruc- 
tion processor reconstructs a volumetric image repre- 
sentation from the generated data lines. The control 
console typically includes one or more monitors 32 and 
an operator input device 34, such as a keyboard, track- 
ball, mouse, or the like. 

[0011] A fluoroscopic imaging system 40 is mechani- 
cally mounted to the planning image scanner 20. More 
specifically to the preferred embodiment, a flat plate x- 
ray detector 42 and an x-ray tube 44 are mounted in a 
cantilevered manner to 180° opposite points on a C-arm 
46. When the x-ray tube is actuated by depressing afoot 
pedal 48, a beam of x-rays is projected through the pa- 
tient support 1 2, through the subject on the patient sup- 
port, and onto the x-ray detector 42. The x-ray detector 
42 converts the x-rays into electronic signals which are 
used to generate human-readable^ real time displays on 
a video monitor 50. Other real time or substantially real 
time imaging systems are also contemplated, such as 
continuous CT (CCT), and the like. 
[001 2] The C-arm 46 is rotatably mounted to a support 



post 52. An encoder in the support post encodes its ro- 
tational position as it is rotated manually or by motor 
around a plane of rotation. A mechanical linkage, such 
as a pair of pivotally interconnected arms or links 54 en- 
s able the fluoroscopic device to be pivoted into the illus- 
trated position and pivoted to the side of the volumetric 
imager for storage. A sliding member 56 is movable 
manually or by motor along one of the arms parallel to 
the plane of rotation. A position encoder encodes the 
10 position of the support along the arm, hence horizontal 
displacement of a centre of rotation of an x-ray source 
and detector. The mechanical interconnections are such 
that the plane of rotation of the fluoroscopic x-ray source 
and detectors is parallel to the plane of rotation of the 
15 CT scanner x-ray source and perpendicular to a longi- 
tudinal axis of the patient support. Moreover, the me- 
chanical interconnection between the fluoroscopic de- 
vice and the CT scanner provides a precisely fixed and 
known offset between the two planes of rotation. 
20 [0013] A minimally invasive planning surgical instru- 
ment 60 is positioned by the operator in monitored po- 
sitions and orientations in preparation for and in carrying 
out a surgical procedure. The surgical planning instru- 
ment in the illustrated embodiment includes a manually 
25 guided biopsy needle 62, although numerous other in- 
struments are contemplated. The position and orienta- 
tion of the instrument are determined by a mechanical 
arm assembly 64. The mechanical arm assembly in- 
cludes a plurality of arm segments which are intercon- 
30 nected by pivot members. Encoders or position resolv- 
ers at each joint monitor the relative articulation and ro- 
tation of the arms relative to each other. Position encod- 
ers or resolvers are also provided along the mechanical 
interconnection of the arm with the CT scanner. In this 
55 manner, the mechanical interconnection which is meas- 
ured by the resolvers and encoders provides an accu- 
rate indication of the position and orientation of the in- 
strument 62 relative to the CT scanner. Because the 
fluoroscopic system is also mechanically constrained to 
40 monitored orientation relative to the CT scanner, the po- 
sition and orientation of the instrument 62 is also known 
relative to the fluoroscopic system. 
[001 4] To verify the relative orientation and position of 
the surgical instrument relative to the patient support, a 
45 tip of the surgical instrument or pointer is touched to the 
calibration marker 16 and an assessment is made 
whether the electronic signals indicative of patient sup- 
port location and surgical instrument location, in fact, 
place both at the same point in space. Analogously, one 
50 or more reference markers 70, 72 are mounted at known 
locations to the fluoroscopic device. Again, touching the 
tip of the surgical instrument or pointer to these markers 
enables verification of their relative positions. Similarly 
one or more markers 74 on the CT scanner in a fixed, 
55 known location relative to the plane of rotation of the x- 
ray beam is touched by the tip of the surgical instrument 
or pointer and the electronic positions signals are com- 
pared to be sure that the coordinate systems of the vol- 
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umetric scanner and the surgical instrument are accu- 
rately correlated. 

[0015] Other nnechanisnns for nnonitoring the position 
of the surgical tool or pointer 62 are also contemplated. 
For example, a plurality of transmitters, such as light 
emitting diodes 80 are mounted in a fixed, l<nown rela- 
tionship to the surgical tool or pointer. An array of re- 
ceivers 82 is mounted in a fixed relationship to the CT 
scanner preferably affixed to the ceiling of the room. 
Each time the emitters are actuated and the emitted sig- 
nal received by the receivers, the position and orienta- 
tion of the planning instrument are accurately and quick- 
ly calculated using geometric triangulation techniques. 
By positioning the planning instrument on a plurality of 
markers, preferably 3 or more, e.g. markers 70 or 72 
which are mounted in a fixed, known relationship to the 
coordinate system of the fluoroscopic scanner, the plan- 
ning instrument coordinate system and the fluoroscopic 
coordinate system can be readily aligned. Analogously, 
by touching the planning instrument to the plurality of 
markers 74 disposed in a known, fixed relationship to 
the coordinate system of the CT scanner, the coordinate 
systems of the planning instrument and CT scanner can 
be readily correlated. Analogously, by positioning the 
planning instrument on marker 16 on the patient table, 
the coordinate system of the planning instrument and 
patient table can be readily coordinated. 
[0016] With reference to Figure 2, an instrument co- 
ordinate circuit 100 determines the position and trajec- 
tory of the instrument 62 in instrument space, particu- 
larly a coordinate system of the instrument. The instru- 
ment coordinate circuit is connected with the resolvers 
on the mechanical arm 64 in the mechanical arm em- 
bodiment and with the receivers 82 in the emitter em- 
bodiment to receive signals indicative of changes of po- 
sition and orientation of the instrument in instrument 
space. An instrument-planning scanner correlating 
processor 102 determines the correlation or transform 
between the instrument 62 and the volumetric scanner 
20 coordinate systems. The correlation or transform is 
normally described in terms of offset (particular offset 
along the axis of the patient support), angular offset or 
rotation, and scaling. In one embodiment, the instru- 
ment 62 is touched to three or more markers 74 which 
are in a known relationship to the volumetric scanner 
coordinate system. By measuring the coordinates of the 
instrument in the instrument coordinate system while 
touching each marker, three or more common points in 
the two coordinate systems are determined. By deter- 
mining a barrycentre, centroid, or other characteristic 
point of the common points, the offset between the two 
coordinate systems is determined. By determining the 
angular difference between the rays from the character- 
istic point to each point in each coordinate system, the 
angular offset is determined. By determining a differ- 
ence in physical displacement between the character- 
istic point and the corresponding points in each coordi- 
nate system, the scaling factor is determined. Of course. 



in a system such as the illustrated embodiment in which 
the instrument and the volumetric scanner are mechan- 
ically linked, the transform or relationship between the 
volumetric scanner and instrument coordinate systems 

s can be predetermined from the mechanical interconnec- 
tion. The touching of markers can be eliminated or used 
merely to confirm that the instrument and CT scanner 
coordinates have not become misaligned. 
[0017] Using analogous mathematics or known me- 

10 chanical relationships, an instrument to fluoroscopic de- 
vice correlation processor 104 determines the correla- 
tion or transform between the instrument and fluoro- 
scopic device coordinate systems. An instrument to pa- 
tient table correlating processor 106 performs a similar 

15 calculation or uses similar known physical relationships 
to determine the correlation or transform between the 
patient table and the instrument. Preferably, a phantom 
having a multiplicity of marks is positioned in a known 
position on the table to provide a larger number of cor- 

20 responding points in both coordinate systems for the 
correlating process. Images of the phantom can be uti- 
lized to derive transforms between patient table space 
and planning or real time image coordinate systems. 
[0018] Resolvers 108 located in the arm 52 of the 

25 fluoroscopic device contribute an angular offset to the 
fluoroscopic device to instrument correlation or trans- 
form when the C-arm including the x-ray source and de- 
tector are rotated and provide a vertical offset when the 
arm 52 is raised or lowered. Analogously, table resolv- 

30 ers 110 located in the patient table contribute vertical 
and longitudinal offsets to the correlation between the 
instrument and the patient table when the table is raised 
or lowered and when the patient support 12 is moved 
axially. A planning to real time, i.e., a volumetric to fluor- 

55 oscopic coordinate system correlation processor 112 
determines a correlation between the volumetric scan- 
ner and the fluoroscopic scanner coordinate systems. 
Because the transforms are linear operators, the trans- 
form between the instrument and volumetric scanner 

40 and the transform between the instrument and the fluor- 
oscopic device coordinate system can be simply com- 
bined. Optionally, an instrument to patient correlation 
processor 114 determines the correlation between the 
instrument coordinate system and a patient coordinate 

45 system. Again, this can be done by placing the instru- 
ment on three or more known reference points on the 
patient. Such points might include readily identifiable 
anatomical structures such as the tip of the nose, dis- 
tinctive points on bones, fiducial markers that are im- 

50 aged during the volumetric orfluoroscopic imaging proc- 
ess, or the like. 

[0019] An instrument to volumetric image coordinate 
system transform processor 120 receives the correla- 
tion or transform from the instrument to planning image 
55 processor 102. The instrument to volumetric image 
processor operates on input position and orientation co- 
ordinates in image space to transform them into volu- 
metric image data space or vice versa. Knowing the po- 
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sition of the instrument in volumetric or planning data 
space enables the instrument position and orientation 
to be superimposed on the volumetric planning image 
data. Analogously, a planning image to real time image 
transform processor 122 operates on input coordinates 
in one of the planning and fluoroscopic coordinate sys- 
tems to generate the corresponding position and trajec- 
tory coordinates in the other. 

[0020] During a medical procedure, the patient is po- 
sitioned in the volumetric planning scanner and a volu- 
metric image is generated. The volumetric image is 
stored in a volumetric or planning data memory 130. The 
position of the patient table during the generation of the 
planning data, particularly as the table moves to gener- 
ate spiral or slice data, is stored in conjunction with the 
volumetric planning data such that the data is correlated 
with the patient table coordinate system. The operator 
control 30 controls the volume planning image data 
memory or a video processor 132 such that selected 
slices, projection images, surface renderings, or other 
conventional displays of the data are generated for dis- 
play on a planning image display 134. Preferably, the 
planning image display includes corresponding sagittal, 
coronal, and traverse slices through a common point of 
intersection. Analogously, the real time fluoroscopic x- 
ray tube 44 is actuated to generate an x-ray beam that 
is detected by the x-ray detector 42 to generate a real 
time image which is stored in a real time image memory 
140. Avideo processor 142 converts the realtime image 
into appropriate format for display on the real time fluor- 
oscopic image display 50. 

[0021] The fluoroscopic device's x-ray beam has a 
centre line that can be aligned with a target in the patient. 
The surgical instrument is positioned on and inserted 
along the centre line while watching the volume display 
in three-dimensional space or the fluoroscopic display 
in two-dimensional space. 

[0022] Because the fluoroscopic display is generated 
during the surgical procedure, the instrument is dis- 
played in the fluoroscopic image. The coordinates and 
trajectory of the instrument are conveyed to the instru- 
ment to planning image transform processor 120 for 
conversion into the planning image coordinate system. 
The location and trajectory of the instrument in the plan- 
ning image coordinate system is communicated to the 
video processor 132 which superimposes the surgical 
instrument position and trajectory on the CT data dis- 
play. The operator control 34, preferably a trackball or 
mouse, generates cursor position signals which are 
transformed into the planning image coordinate system 
144 and communicated to the video processor 132 to 
generate a movable cursor point on the planning image 
display 134. The same coordinates are also communi- 
cated to the planning image to real time image transform 
processor 1 22 to be transformed into the real time image 
coordinate system and to the video processor 1 42 to be 
superimposed on the fluoroscopic image. In this man- 
ner, the cursor is positioned at like points in both dis- 



plays. 

[0023] In one embodiment, a projection processor 
150 projects the volumetric image data from the plan- 
ning image memory 130 along selected orientations. 

s More specifically, the projection processor 150 is con- 
nected with the planning to real time (volumetric to fluor- 
oscopic) coordinate system correlation processor 112 
or the fluoroscopic device resolvers 108 in order to gen- 
erate the same projection as is currently displayed on 

10 the fluoroscopic display 50. An image superimposition 
or summing processor 152 superimposes, sums, or oth- 
erwise combines the projected volumetric image and 
the fluoroscopic image and provides these summed im- 
age signals to the video processor 142 for display on 

15 the monitor 50. In this manner, higher and lower resolu- 
tion projection images are combined to improve the ap- 
parent resolution. As the C-arm is rotated, the projection 
processor 1 50 reprojects the volumetric image along the 
new direction. Alternately, the two images can be dis- 

20 played side by side, subtracted, or the like. 

[0024] In another embodiment, during a surgical pro- 
cedure, the instrument is moved to a location in the pa- 
tient while monitoring the fluoroscopic image. To check 
its exact position, the C-arm 46 is rotated several de- 

25 grees. preferably about 90° and a second fluoroscopic 
image 140' is generated. A position analyzing processor 
160 analyzes the two angularly offset fluoroscopic im- 
ages to determine the position and orientation of the in- 
strument in three dimensions in the coordinate system 

30 of the fluoroscopic device. As illustrated by way of ex- 
ample in Figure 3, a point 162 in the patient is projected 
as a spot 164 in the first fluoroscopic image and a spot 
164' in the second fluoroscopic image. The spot 164 is 
projected along a ray 166 back to the origin at the x-ray 

35 source. Analogously spot 164' is projected back along 
a ray 166' to the origin or x-ray source. These two rays 
intersect at the point 162, thus identifying the coordi- 
nates of the point 162. A large plurality of projection im- 
ages can be backprojected to identify the coordinate. It 

40 will be appreciated that Figure 3 illustrates only a single 
slice of the patient. However, because the projection im- 
ages are two-dimensional, a multiplicity of such slices 
are analyzed concurrently to generate the coordinates 
in three dimensions in fluoroscopic image space. The 

45 coordinates indicative of the position and trajectory of 
the instrument are communicated to the volumetric to 
fluoroscopic coordinate system transform processor 
122 to be converted into corresponding coordinates in 
planning volume image space and to the video proces- 

50 sor 122 such that the surgical instrument is superim- 
posed on the planning image data. 
[0025] In another alternate embodiment, the cursor is 
positioned at the point of intersection of concurrently dis- 
played transverse, coronal, and sagittal views on the 

55 volumetric image display 134. As the operator moves 
the cursor control 34 through volumetric image data 
space, the sagittal, coronal, and transverse views 
change corresponding. When the cursor is also dis- 



5 



9 



EP 0 930 046 A2 



10 



played on the fluoroscopic display, moving the cursor 
through the fluoroscopic image causes the sagittal, 
coronal, and transverse views on the volumetric image 
display to change correspondingly. 
[0026] It is to be appreciated that numerous other 
techniques can be utilized for correlating the above-dis- 
cussed coordinate systems. For example, when two 
fluoroscopic images are generated, the images can be 
analyzed to determine the coordinates of fiducial mark- 
ers, characteristic anatomical structures, or the like in 
fluoroscopic image space. The same points can then be 
identified in volume image space. An analogous corre- 
lation process to that discussed above can then be uti- 
lized to correlate fluoroscopic image and volumetric da- 
ta space. When common fiducial markers are used in 
both the volumetric image and the fluoroscopic image, 
the fluoroscopic and volumetric images can be analyzed 
or windowed electronically to identify the corresponding 
markers. In this manner, the corresponding markers can 
be identified and correlated automatically without oper- 
ator intervention. A similar operation can be done with 
anatomical markers, if the anatomical markers can be 
distinguished by the computer without operator inter- 
vention. Alternately, the procedure can be made semi- 
automatic requiring the operator only to identify the cor- 
responding anatomical structures in all images. 
[0027] Among advantages of the described embodi- 
ments are that surgery can be monitored in real time; 
surgical instrument position can be accurately com- 
bined with previously generated volumetric diagnostic 
images; and real time images can be integrated with 
previously collected diagnostic images. 



Claims 

1 . A method of imaging in which a region of interest of 
a patient is imaged using a three-dimensional diag- 
nostic imaging technique to generate volumetric im- 
age data, as a surgical instrument is inserted into 
the region of interest, a series of fluoroscopic pro- 
jection images of the region of interest are generat- 
ed including the surgical instrument for monitoring 
movement of the surgical instrument into the region 
of interest, the volumetric image data being gener- 
ated and displayed in a volumetric image coordi- 
nate system and the fluoroscopic images being 
generated and displayed in a fluoroscopic image 
coordinate system, and further including: electron- 
ically correlating the fluoroscopic image coordinate 
system and the volumetric image coordinate sys- 
tem. 

2. A method as claimed in claim 1, further including: 
monitoring the position of the surgical instrument in 
the fluoroscopic coordinate system; transforming 
the position of the surgical instrument from the fluor- 
oscopic image coordinate system into the volumet- 



ric image coordinate system; and displaying an in- 
dication of the surgical instrument in the volumetric 
image coordinate system superimposed on at least 
one volumetric image display. 

5 

3. A method as claimed in claim 1 and 2, further in- 
cluding: generating a fluoroscopic image display; 
generating a volumetric image display; generating 
a cursor display on one of the fluoroscopic and vol- 

^0 umetric image displays; transforming a coordinate 
location of the cursor from the coordinate system of 
the one display into the coordinate system of the 
other display and displaying the cursor at the cor- 
responding coordinate position in the other display 

^5 such that the cursor is displayed at corresponding 
coordinate positions in both displays. 

4. A method as claimed in any one of claims 1 to 3, 
further including: generating at least two fluoro- 

20 scopic projection images which are angularly dis- 
placed from each other; analyzing the angularly dis- 
placed fluoroscopic projection images to determine 
at least one of a position and trajectory of the instru- 
ment in the fluoroscopic image coordinate system; 

25 transforming the instrument location or trajectory 
from the fluoroscopic image coordinate system to 
the volumetric image coordinate system; superim- 
posing at least one of the instrument trajectory and 
position on the volumetric image. 

30 

5. A method as claimed in one of claims 1 to 4, further 
including: projecting the volumetric image to gener- 
ate a projection image; transforming the projection 
image from the volumetric image coordinate system 

55 into the fluoroscopic image coordinate system; and 
displaying the transformed projection image in as- 
sociation with the fluoroscopic image. 

6. A method as claimed in any one of claims 1 to 5, 
40 wherein the correlating step includes: correlating 

the volumetric image coordinate system with a co- 
ordinate system of an instrument to determine a vol- 
umetric image space/instrument space transform 
therebetween; correlating the fluoroscopic image 

45 coordinate system with the instrument coordinate 
system to generate a fluoroscopic image space/in- 
strument space transform therebetween; combin- 
ing the volumetric image space/instrument space 
transform with the fluoroscopic space/instrument 

50 space transform to generate a fluoroscopic space/ 
volumetric image space transform for transforming 
directly between the fluoroscopic image coordinate 
system and the volumetric image coordinate sys- 
tem. 

55 

7. A method as claimed in claim 6, further including: 
operating on a coordinate position of an instrument 
in instrument space with the volumetric image 
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space/instrument space transform to generate a 
corresponding position coordinate in tlie volunnetric 
image coordinate system; superimposing a display 
of the transformed coordinate on a display of the 
volumetric image data; generating a human-reada- 
ble display of the fluoroscopic image data; generat- 
ing a human-readable display from the volumetric 
image data; superimposing a cursor on one of the 
fluoroscopic and volumetric image displays; trans- 
forming coordinates of the cursor using the fluoro- 
scopic space/volumetric image space transform 
and generating a corresponding cursor display on 
the other image; under operator control, moving the 
cursor coordinately in both images. 

8. A method as claimed in any one of claims 1 to 7, 
further including: aligning a centre line of the fluor- 
oscopic image with a target in the patient while dis- 
playing the position of this centreline on the volume 
display; inserting the surgical instrument along the 
centre line. 

9. Medical imaging apparatus comprising a three-di- 
mensional diagnostic imaging apparatus for gener- 
ating three-dimensional diagnostic image data 
which is indicative of anatomical structure of a re- 
gion of interest in which a patient is imaged, means 
for generating a series of fluoroscopic projection im- 
ages of the region of interest including of an inserted 
surgical instrument for monitoring movement of the 
surgical instrument into the region of interest, the 
volumetric image data being generated and dis- 
played in a volumetric image coordinate system and 
the fluoroscopic images being generated and dis- 
played in a fluoroscopic image coordinate system 
in use, and further including means for electronical- 
ly correlating the fluoroscopic image coordinate 
system and the volumetric image coordinate sys- 
tem. 

10. Medical imaging apparatus comprising: a surgical 
instrument (62) which is movable by an operator to 
any of a plurality of locations adjacent and in a re- 
gion of interest of a subject; a system (1 00) for gen- 
erating electrical signals indicative of coordinates of 
the instrument in instrument space; a three-dimen- 
sional diagnostic imaging apparatus (20) for gener- 
ating three-dimensional diagnostic image data 
which is indicative of anatomical structure of the re- 
gion of interest in three-dimensional image space; 
a th ree-dimensional diagnostic image data memory 
(130) for storing the generated three-dimensional 
diagnostic image data; a real time imaging device 
(40) for generating real time image data of the re- 
gion of interest of the subject and the medical in- 
strument when the medical instrument is positioned 
in or adjacent the region of interest in real time im- 
age space; a transform processor (102, 120, 122) 



for transforming at least one coordinate of the sur- 
gical instrument into the three-dimensional volu- 
metric image space; at least one human-readable 
display (134) connected with the transform proces- 
sor, the three-dimensional image data memory 
(1 44) and the real time imaging device (40) for gen- 
erating a human-readable display of a real time im- 
age and at least one of (i) slices of the three-dimen- 
sional imaging data which slices intersect at the 
transformed coordinate and (ii) an image generated 
from the three-dimensional image data with a rep- 
resentation of the surgical instrument superim- 
posed thereon. 

11. Apparatus as claimed in claim 10, wherein the real 

time imaging device includes a C-arm (46) mounted 
fluoroscopic device (42) which generates fluoro- 
scopic projection images and further including: first 
and second fluoroscopic image memories (140, 
1 40') for storing fluoroscopic projection images tak- 
en at different angles; a processor (160) for analyz- 
ing the fluoroscopic projection images to determine 
coordinates of a selected point in fluoroscopic im- 
age space, which determined coordinates are sup- 
plied to the transform processor (122) to be trans- 
formed into corresponding coordinates in three-di- 
mensional image space. 
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